Analyzing data from single-case alternating treatments designs

Alternating treatments designs (ATDs) have received comparatively less attention than other single-case experimental designs in terms of data analysis, as most analytical proposals and illustrations have been made in the context of designs including phases with several consecutive measurements in the same condition. One of the specific features of ATDs is the rapid (and usually randomly determined) alternation of conditions, which requires adapting the analytical techniques. First, we review the methodologically desirable features of ATDs, as well as the characteristics of the published single-case research using an ATD, which are relevant for data analysis. Second, we review several existing options for ATD data analysis. Third, we propose 2 new procedures, suggested as alternatives improving some of the limitations of extant analytical techniques. Fourth, we illustrate the application of existing techniques and the new proposals in order to discuss their differences and similarities. We advocate for the use of the new proposals in ATDs, because they entail meaningful comparisons between the conditions without assumptions about the design or the data pattern. We provide R code for all computations and for the graphical representation of the comparisons involved. (PsycINFO Database Record

Defining and assessing immediacy in single-case experimental designs

Immediacy is one of six data aspects (alongside level, trend, variability, overlap, and consistency) that has to be accounted for when visually analyzing single-case data. Given that it is one of the aspects that has received considerably less attention than other data aspects, the current text offers a review of the proposed conceptual definitions of immediacy (i.e., what it refers to) and also of the suggested operational definitions (i.e., how exactly is it assessed and/or quantified). Provided that a variety of conceptual and operational definitions is identified, we propose following a sensitivity analysis using a randomization test for assessing immediate effects in single-case experimental designs, by identifying when changes were most clear. In such a sensitivity analysis, the immediate effects are tested for multiple possible intervention points and for different possible operational definitions. Robust immediate effects can be detected if the results for the different operational definitions converge

The Single-Case Data Analysis Package: Analysing Single-Case Experiments with R Software

The RcmdrPlugin.SCDA plug-in package is discussed. It integrates three R packages in the R commander interface: SCVA (for Single-Case Visual Analysis), SCRT (for Single-Case Randomization Tests), and SCMA (for Single-Case Meta-Analysis). This way the plug-in package covers three important steps in the analysis of single-case data

A Parametric Bootstrap Version of Hedges’ Homogeneity Test

Hedges’ Q-test is frequently used in meta-analyses to evaluate the homogeneity of effect sizes, but for several kinds of effect size measures it does not always appropriately control the Type 1 error probability. Therefore we propose a parametric bootstrap version, which shows Type 1 error control under a broad set of circumstances. This is confirmed in a small simulation study

The assessment of consistency in single-case experiments: Beyond A-B-A-B designs

Quality standards for single-case experimental designs (SCEDs) recommend inspecting six data aspects: level, trend, variability, overlap, immediacy, and consistency of data patterns. The data aspect consistency has long been neglected by visual and statistical analysts of SCEDs despite its importance for inferring a causal relationship. However, recently a first quantification has been proposed in the context of A-B-A-B designs, called CONsistency of DAta Patterns (CONDAP). In the current paper, we extend the existing CONDAP measure for assessing consistency in designs with more than two successive A-B elements (e.g., A-BA-B-A-B), multiple baseline designs, and changing criterion designs. We illustrate each quantification with published research

The Evaluation Self-Efficacy scale

The Evaluation Self-Efficacy Scale(ESE)was developed as an outcomes assessment instrument for social work courses focusing on evaluation. In the two studies on the scale to date, the results regarding its psychometric properties have been encouraging

An assessment of the predictive validity of impact factor scores: Implications for academic employment decisions in social work

Citation for final version: Holden, G., Rosenberg, G., Barker, K., & Onghena, P. (2006). An assessment of the predictive validity of impact factor scores: Implications for academic employment decisions in social work. Research on Social Work Practice, 16, 6, 613-624.Objective: Bibliometrics is a method of examining scholarly communications. Concerns regarding the utility of bibliometrics in general, and the impact factor score (IFS) in particular, have been discussed across disciplines including social work. While there are frequent mentions in the literature of the IFS as an indicator of the impact or quality of scholars’ work, little empirical work has been published regarding the validity of such use. Method: A proportionate, stratified, random sample, of n=323 articles was selected from 17 Web of Science listed social work journals published during the 1992-1994 period. Results: The relationship between journals’ impact factor scores and the actual impact of articles published in those journals (predictive validity) was r = .41 (short term) and r = .42 (long term). Conclusion: The practice of using the IFS as a proxy indicator of article impact merits significant concern as well as further empirical investigation. The final, definitive version of this article has been published in Research on Social Work Practice, 16, 6 © SAGE Publications Ltd at the Research on Social Work Practice page: http://rswp.sagepub.com/ on SAGE Journals Online: http://online.sagepub.com

Assessing Consistency in Single-Case Alternation Designs

Consistency is one of the crucial single-case data aspects that are expected to be assessed visually, when evaluating the presence of an intervention effect. Complementarily to visual inspection, there have been recent proposals for quantifying the consistency of data patterns in similar phases and the consistency of effects for reversal, multiple-baseline, and changing criterion designs. The current text continues this line of research by focusing on alternation designs using block randomization. Specifically, three types of consistency are discussed: consistency of superiority of one condition over another, consistency of the average level across blocks, and consistency in the magnitude of the effect across blocks. The focus is put especially on the latter type of consistency, which is quantified on the basis of partitioning the variance, as attributed to the intervention, to the blocking factor or remaining as residual (including the interaction between the intervention and the blocks). Several illustrations with real and fictitious data are provided in order to make clear the meaning of the quantification proposed. Moreover, specific graphical representations are recommend for complementing the numerical assessment of consistency. A freely available user-friendly webpage is developed for implementing the proposal

Effect of secondary schools on academic choices and on success in higher education

Peer reviewe

Pediatric pain and anxiety: A meta-analysis of outcomes for a behavioral telehealth intervention

The utility of using single system designs in an optimal set of circumstances is considered. The effects of a computer network intervention on three outcomes (pain intensity, pain aversiveness and anxiety) were explored in previous evaluations via restricted alternating treatment designs. This paper presents an overview of the results of those evaluations, along with new metaanalyses that combine results across all three studies and explore potential moderating variables. This is followed by a discussion of the utility of this approach for social work research. Final version of the manuscript for citation: Holden, G., Bearison, D., Rode, D., Fishman-Kapiloff, M., Rosenberg, G. & Onghena, P. (2003). Pediatric pain and anxiety: A meta-analysis of outcomes for a behavioral telehealth intervention. Research on Social Work Practice, 13, 693-704